С52315 
Algorithm Fundamentals 


Searching Algorithms 


Lectures prepared by: Dr. Manal Alharbi, and Dr. Areej Alsini 


Review: Searching Problem 


e Assume A is an array with n elements А[1], A[2], ... A[n]. For a 
given element x, we must determine whether there is an index J; 
1< у < п, such that х= А[/] 


- Two algorithms, among others, address this problem 
- Linear Search 
* Binary tree: Binary Search 


Linear Search Algorithm 


Algorithm: LINEARSEARCH 
Input: array A[1..2] of п elements and an element x. 


Output: / if x= АЛ, 1 € /x n, and О otherwise. 


1. j=l 
2. while (/< n) and (xz A[j]) do 
3. J=jt 1 
4. end while 
5. if x-A[j] then return | else return 0 


Analyzing Linear Search 


* One way to measure efficiency is to count how many statements get executed before the 


algorithm terminates 
* One should keep an eye, though, on statements that are executed "repeatedly". 


* What will be the number of “element” comparisons if x 
* First appears in the first element of A 
* First appears in the middle element of A 
* First appears in the last element of A 


* Doesn't appear in A. 


Analyzing Linear Search 


* One way to measure efficiency is to count how many statements get executed before the 


algorithm terminates 
* One should keep an eye, though, on statements that are executed "repeatedly". 


* What will be the number of “element” comparisons if x 


* First appears in the first element of A 1 comparison 
* First appears in the middle element of A = comparisons 
* First appears in the last element of A п comparisons 


* Doesn't appear in A. п comparisons 


Analyzing Linear Search 


We are interested here in finding the number of element comparisons .In the algorithm, element 
comparisons are done in the while statement (Step 2) of the algorithm, that is 


2. while (5 < n) and (хя A[j/) 


The minimum number of comparisons occurs when the element we are looking for is the first element in 
the array. In this case, there is only one element comparison. 


The maximum number of element comparisons occurs when the element is not among the elements of 
the array or in the last index. This results in n comparisons. 


Thus, the number of element comparisons in the linear search algorithm is between 1 and n inclusive. 


Linear Search Algorithm 


Algorithm: LINEARSEARCH 
Input: array A[1..2] ої n elements and an element x. 
Output: / if x= A[], 1 € /x n, and 0 otherwise. 


| j-1 
while (/ < п) and (xz A[j]) do 
J=j+1 


end while 
if x=A[j] then 
return j 
else return 0 


* K ranges between 1 апа п. 
e Worst-case time n- 1 = 2п – 1 
* Total time complexity = O(2n — 1) = O(n) 


Can we do better? Review: binary tree- concept 


- Yes, using binary search. 


* Algorithms Design: Divide-and-conquer 


e Let л be any problem with |л| = n. To solve using divide-and-conquer the following steps 
are involved: 
e Generate k subproblems form л (for some К> 1). 
• Let these subproblems be т,, Ty... ту 
* fori-1tokdo 
e Recursively (or otherwise) solve т; 
* Combine the solutions obtained in step 2 to create a solution for т. 


* Aclassical example of divide-and-conquer is binary search. For this problem, the input are a sorted 
sequence X = К,, сл... Ка and another element x. The problem is to check if x € X. 


Binary Search Trees- Review 


* Tree representation: 
* A linked data structure in which each node 15 an object 


L 
і 


* Node representation: 
* Key field 
* Left: pointer to left child 
* Right: pointer to right child 
* p: pointer to parent (p [root [T]] = NIL) 


- Satisfies the binary-search-tree property 1! 


Binary Search Tree Property 


* Binary search tree property: 


• If y is in left subtree of x, 
then key [y] < key [x] 


• If y is in right subtree of x, 
then key [y] » key [x] 


Binary Search Trees 


• Support many dynamic set operations 


* SEARCH, MINIMUM, MAXIMUM, PREDECESSOR, SUCCESSOR, 
INSERT, DELETE 


* Running time of basic operations on binary search trees 
* On average: O(logn) 
* The expected height of the tree is logn 


* [n the worst case: O(n) 


* The tree is a linear chain of n nodes 


Worst Case 


- If the tree is very unbalanced, then running time will be О(п). 


N 
worst case lat 
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^^ һ=п-1 


/ (no better than a linked list) 


LN 
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Traversing a Binary Search Tree 


* Inorder tree walk: 
Root is printed between the values of its left and right subtrees left, root, right 
e Keys are printed in sorted order 


Example: 
e Preorder tree walk: 
, | | Inorder:235579 
* root printed first: root, left, right (5) 
Preorder: 532579 
© Q 


e Postorder tree walk: 
* root printed last: left, right, root 2 O (9) Postorder: 253975 


Traversing a Binary Search Tree 


Alg: INORDER-TREE-WALK(x) 


1. 


2 
3. 
4 


if x = NIL 
then INORDER-TREE-WALK ( left [x]) 
print key [x] 
INORDER-TREE-WALK ( right [x]) 


Тэ 
Output: 235579 


Running time: 


@(n), where n is the size of the tree rooted at x 


Searching for a Key 


* Given a pointer to the root of a tree and a key К: 


e Return a pointer to a node with key К 


if one exists 


e Otherwise return NIL 


• Idea 


Starting at the root: trace down a path by comparing k with the key of the current node: 
e If the keys are equal: we have found the key 
If K key[x] search in the left subtree of x 
If k > key[x] search in the right subtree of x 


Example: TREE-SEARCH 


- Search for key 13: 
(6) (18) e 155657513 


Searching for a Key 


Ай: TREE-SEARCH(x, К) 


if x = NIL or k = key [x] 
then return х 

if k < key [x] 
then return TREE-SEARCH(left [x], k ) 
else return TREE-SEARCH(right [x], k ) 


Running Time: O (h), 
h — the height of the tree 


E P Nm 


Finding the Minimum 
in a Binary Search Tree 


e Goal: find the minimum value in a BST 
* Following left child pointers from the root, until а NIL is encountered 


Ай: TREE-MINIMUM(x) 


1. while left [x] + NIL (15) 
2. do x < left [x] є 15 
3. return X BY Ma & 
Q) (0 Q3 
2 


Running time: O(h), h — height of tree 


Minimum = 2 


Finding the Maximum 
in a Binary Search Tree 


e Goal: find the maximum value in a BST 
* Following right child pointers from the root, until a NIL is encountered 


Allg: TREE-MAXIMUM (x) 
1. while right [x] # NIL 


2. do x — right [x] 15) 

3. return X © @) 
з) 0ш 00 

Running time: O(h), h — height of tree J (9 £z 


Maximum - 20 


Successor 


Def: successor (х) =Y, such that key [y] is the 
smallest key > key [x] 


• E.g.: successor(15) 17 
successor (13) = 15 
successor (9) = тз 


* Case 1: right (x) is nonempty 
* successor (x) = the minimum in right (X) 
* Case 2: right (X) is empty 
* go up the tree until the current node is a left child: 
successor (x.) 18 the parent of the current node 


* if you cannot go further (and you reached the root): 
X is the largest element 


Finding the Successor 


Alg: TREE-SUCCESSOR (x) 

1. ifright [x] + NIL 

2 then return TREE-MINIMUM(right [x]) 
3. y <= plx] 

4. while y = NIL and x = right [y] 

5 do x - y 

6 y — ply] 

7 


return y 


Running time: О (h), h — height of the tree 


Predecessor 


Def: predecessor (x) = у, — that " Ly] is the 
biggest key « e 
• E.g.: predecessor (15) = 13 
predecessor (9)= 7 
predecessor (7)= 6 


* Case I: left (x) is nonempty 
* predecessor (x) = the maximum in left (x) 
* Case 2: left (x) is empty 
* go up the tree until the current node is a right child: 
predecessor (x) is the parent of the current node 


- if you cannot go further (and you reached the root): 
X is the smallest element 


Insertion 


* Goal: 


* Insert value v into a binary search tree 


* Idea: 
* If key [x]< v move to the right child of x, 


Insert value 13 


else move to the left child of X 


* When X is NIL, we found the correct position 


• [f v < key [y] insert the new node as y's left child 


else insert it as y's right child 


* Begining at the root, go down the tree and maintain: 
* Pointer X : traces the downward path (current node) 


* Pointer y : parent of X ("trailing pointer" ) 


Example: TREE-INSERT 


Insert 13: Ф 22 x=root[T], yz NIL — y 


Alg: TREE-INSERT(T, 2) 


ив озь 


10. 
11. 
12. 
13. 


у — NIL 
х — root [T] 
while х # NIL 
do y < х 
if key (21 « key [x] 
then x «- left [x] 
else x — right [x] 
plz] — у 
if y - NIL 
then root [T] — z 
else if key [z] « key [y] 
then left [y] — z 
else right [y] — z 


Tree was empty 


Running time: O(h) 


Deletion 
* Goal: 


- Delete a given node 7 from a binary search tree 


* Idea: 


e Case 1: 2 has no children 
* Delete 2 by making the parent of 2 point to NIL 


Deletion 
- Case 2: 2 has one child 


- Delete 2 by making the parent of Z point to 25 child, 
instead of to Z 


Deletion 
* Case 3: 2 has two children 
* Z's successor (y) is the minimum node in Z's right subtree 
* y has either no children or one right child (but no left child) 
* Delete y from the tree (via Case 1 or 2) 
* Replace z's key and satellite data with y's. 


(16) (D 
cu NS 
рл „ s 


delete 


TREE-DELETE(T, 7) 


1. if left[z] = NIL or right[z] = NIL 
then у - 2 z has one child 
else у — TREE-SUCCESSOR(Z) z has 2 children 

if left[y] + NIL 


then x — left[y] ОС 


$^ 3 
else x — right[y] 


if x + NIL 
Ө 
. then p[x] — ply] 


9 з Q t RF WN 


TREE-DELETE(T, z) — cont. 


9. if ply] = NIL 


10. then root[T] — x ^ Е 5. 
11. elseif y = left[p[y]] * 


12. then left[p[y]] — x 
13. else right[p[y]] — x 
14. ify zz 


15. then key[z] — key[y] 
16. copy y’s satellite data into z 


17. return y Running time: O(h) 


Binary Search Trees - Summary 


* Operations on binary search trees: 


SEARCH Och) 
e PREDECESSOR O(h) 
. SUCCESOR O(h) 
* MINIMUM O(h) 
* MAXIMUM O(h) 
* INSERT O(h) 
„DELETE Och) 


- These operations are fast if the height of the tree 15 
small — otherwise their performance is similar to that 
of a linked list 


Theorem A R 


Со о с С 


* The number of comparisons performed by Algorithm BINARY SEARCH on a sorted array 
of size n is at most |[og n| + 1 


* Proof: 


* Fact: If T is a binary tree with n nodes whose height is h, then, 


max 


In a binary tree the maximum number of nodes we can have in level / is 2*!. 
* This implies that the maximum number of nodes we can have in a binary tree of height Ais 1 +2 4. + 2^ 2^— J. 
e [n other words, n €2^— 1 5 log; n zh-1 h log; n+l 


* The fact that A < nis easy to see. This happens when we have а skewed tree where each node (other than the leaf) has a single 
child. 


* Abinary tree is said to be fu//if every non-leaf has exactly two children and all the leaves are at the same level. A binary tree is 
defined to be complete if it is full except that there could be some nodes missing in the last level and the missing nodes in the last 
level (if any) are right justified. The height of a complete binary tree with п nodes is O(logz). 


Binary Search 


* We can do “better” than linear search if we knew that the elements of A are sorted, say in non-decreasing order 


(increasing order). 


* As stated above, we are interested in determining whether a given element is among the list of elements stored in the 
array of size n. 
* When the elements are sorted, we use a more efficient algorithm such as the binary search algorithm as follows: 


e Let A/low.. high] be a non-empty array of elements sorted in nondecreasing order and let А/пиа/ be the middle 


element. 
e The idea is that you can compare x to the middle element of A, say A[middle |]. 
If x^ A/mid]: 
e We observe that If xis in A, then it must be one of the elements Amid I, А/пиа--2/,...., Amid high] 
e It follows that we only need to search for in Amid 1... high]. 
In other words, the entries in A//ow...znid] are discarded. 


e Similarly, if x< A/mid], then we only need to search for in A//ow...muid-1] 


* This results in an efficient strategy, which is referred to as binary search. 


Example 1. In this example, we will search for the element x = 35 or any value > 35 
in an array of 14 elements. The array is assumed to be sorted in nondecreasing order 
with the values as shown. 


Index: 1 2 3 4 5 6 1 12 13 14 


«ла 14157815 | [ш зэ Га Га а [зв] 


The execution of the binary search algorithm on the above array proceeds as follows: 


First, the index (referred to as mid in Algorithm BINARYSEARCH) of the array 
element to be compared with the value x is computed using the equation: 


mid = |(low + high) /2| = |(1+14)/2| = 7 


and the value x (=35) is compared with A[7] (= 10) and since 35 > A[7], the portion 
of the array АГ .. 7] is discarded. 


Example 1 (Cont.) 


Thus, the remaining portion of the array to be searched next is now reduced to the 
following subarray: 


Index: 8 9 10 11 12 11 14 


Next, we repeat the process of computing the index of the array element (among 
the remaining elements of the subarray A[8..14] ) to be compared with x. In this 
case, we use low = 8 and high =14, and get: 


mid |(low+high) /2| = |(8+14)/2|=11 


Then x is compared with A[11] (= 23) and since 35 > A[11], the portion of the array 
A[8..11] is discarded. 


* Example 1 (Cont.) 
* Finally, x is compared with A[14] (235) as 


mid = |(14 + 14)/2] = 14 


and hence the search is successively completed. 


Binary Search Algorithm 


Algorithm: BINARY SEARCH 


Input: An array A[1..7] of n elements sorted in non-decreasing order 
and an element x. 


Output: / if x= A[j], 1 € /x n, and 0 otherwise. 
» Ой lr NMOL = пе у = 9 
. While (low 2 high) апа (7 = 0) 


mid = | (Том + high) /2] 
if x = A[mid] then 7 = mid 
else if x < А|тід| then Aigh = mid - 1 


else /о = mid+ 1 Can be considered 
as опе comparison 


. end while 


. return 7 


